---

title: Evasion and Hardening of Tree Ensemble Classifiers

keywords: fastai
sidebar: home_sidebar

summary: "This project tries to implement the idea mentioned in the <a href='https://arxiv.org/pdf/1509.07892.pdf'>paper</a> which is to find finding for a given instance `x` the “nearest” instance `x_prime` such that the classifier predictions of `x` and `x_prime` are different."
description: "This project tries to implement the idea mentioned in the <a href='https://arxiv.org/pdf/1509.07892.pdf'>paper</a> which is to find finding for a given instance `x` the “nearest” instance `x_prime` such that the classifier predictions of `x` and `x_prime` are different."
---
<!--

#################################################
### THIS FILE WAS AUTOGENERATED! DO NOT EDIT! ###
#################################################
# file to edit: index.ipynb
# command to build the docs after a change: nbdev_build_docs

-->

<div class="container" id="notebook-container">
        
    {% raw %}
    
<div class="cell border-box-sizing code_cell rendered">

</div>
    {% endraw %}

<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>Currently we have looked at Random Forest Classifiers ( scikit-learn ) implementation but the idea can easily be extended to GBDT. Currently the package implements the <code>Approximate Evasion</code> method implemented in the paper. For experimental evaluation we look at <code>MNIST</code> digit classification dataset with only two categories <code>2</code> and <code>6</code>. The paper also chooses this dataset because it is well studied datasets, one-to-one mapping between pixels and features and features can vary independently of each other. We can pictorially represent evading instances, and this helps understanding the models’ robustness or lack of.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="How-to-use">How to use<a class="anchor-link" href="#How-to-use"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">

<pre><code>from sklearn.ensemble import RandomForestClassifier

from tree_evasion.core import *
from tree_evasion.tree import *
from tree_evasion.symbolic_prediction import *

Xtr, Xva, Xte, ytr, yva, yte = get_mnist_dataset(SEED)

clf = RandomForestClassifier(n_estimators=5, max_depth=4, random_state=SEED, n_jobs=-1)
clf.fit(Xtr, ytr)

# performance on the holdout set
print(clf.score(Xva, yva))

pairs = CoordinateDescent.get_pairs(clf, Xte)</code></pre>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>In this example we want to change only one pixel for an instance and see if the classifier prediction changes or not. <a href="02_SymbolicInstance.ipynb">notebook</a></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Results">Results<a class="anchor-link" href="#Results"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><img src="/tree_evasion/images/example_1.png" alt=""></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><img src="/tree_evasion/images/example_2.png" alt=""></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p><img src="/tree_evasion/images/example_3.png" alt=""></p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Applications">Applications<a class="anchor-link" href="#Applications"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<p>As the paper mentions, we expect a high performance learning algorithm to generalize well and be hard to evade: only a “large enough” perturbation δ should be able to alter its decision. The existence of small-δ evading instances shows a defect in the generalization ability of the model, and hints at improper model class and/or insufficient regularization.</p>
<p>Since machine learning systems are being deployed in many security-oriented applications. In these applications attacker has a large incentive to find evading instances and fool the system.</p>

</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<h2 id="Next-Steps">Next Steps<a class="anchor-link" href="#Next-Steps"> </a></h2>
</div>
</div>
</div>
<div class="cell border-box-sizing text_cell rendered"><div class="inner_cell">
<div class="text_cell_render border-box-sizing rendered_html">
<ul>
<li>[ ] We have only tackled the <code>Approximate Evasion</code>, paper mentions another algorithm which uses Mixed Integer Linear Programming.</li>
<li>[ ] After finding good evading instances, author suggests to include them in the training set and retrain. This process is called <code>Adversarial Boosting</code>.</li>
</ul>

</div>
</div>
</div>
</div>
 

